Skip to content

Add support for async io #2608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 422 commits into
base: main
Choose a base branch
from
Open

Add support for async io #2608

wants to merge 422 commits into from

Conversation

@sfc-gh-turbaszek
Copy link
Contributor

@sfc-gh-turbaszek sfc-gh-turbaszek commented Oct 28, 2025

Please answer these questions before submitting your pull requests. Thanks!

  1. What GitHub issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    Fixes #NNNN

  2. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am modifying authorization mechanisms
    • I am adding new credentials
    • I am modifying OCSP code
    • I am adding a new dependency

  3. Please describe how your code solves the related issue.

    Please write a short description of how your code change solves the related issue.

  4. (Optional) PR for stored-proc connector:

sfc-gh-fpawlowski and others added 30 commits September 3, 2025 15:52
@sfc-gh-fpawlowski @sfc-gh-pczajka
[Async] Apply #2240 to async code
4d3f07f
@sfc-gh-mkubik @sfc-gh-pczajka
NO-SNOW skip tests of custom requestId on olddriver (#2256)
92cfbc0 
(cherry picked from commit cc563d8)
@sfc-gh-fpawlowski @sfc-gh-pczajka
[Async] Apply #2256 to async code
647a517
@sfc-gh-pczajka
Fix #2227 async implementation
deecc56
@sfc-gh-jszczerbinski @sfc-gh-fpawlowski
SNOW-2026002: Change invalid TLD to be RFC compliant (#2288)
b704b0e 
(cherry picked from commit ce85800)
@sfc-gh-mmishchenko @sfc-gh-fpawlowski
SNOW-2055494 fix proper boto min versions (#2295)
c4e99d9 
(cherry picked from commit f80d83e)
@sfc-gh-mmishchenko @sfc-gh-fpawlowski
SNOW-2057797 Minor python connector version bump (#2302)
989750c 
(cherry picked from commit 98381ae)
@sfc-gh-mmishchenko @sfc-gh-fpawlowski
SNOW-2057797 Update requirements files (#2305)
2a8a146 
Co-authored-by: github-actions <github-actions@github.com>
(cherry picked from commit 15efed3)
@sfc-gh-nkumar @sfc-gh-fpawlowski
SNOW-2052629: Add basic arrow support for Interval data types (#2296)
24da50c 
(cherry picked from commit bbc2f80)
@sfc-gh-fpawlowski
[Async] Apply #2296 to async code
b61cd0c
@sfc-gh-mkubik @sfc-gh-fpawlowski
NO-SNOW Enable structured types in fdn tables to unblock the CI (#2313)
8dd8c0c 
(cherry picked from commit b866808)
@sfc-gh-fpawlowski
[Async] Apply #2313 to async code
3291ffb
@sfc-gh-pbulawa @sfc-gh-fpawlowski
SNOW-1959514: Pandas single quote character fix (#2307)
69765b5 
(cherry picked from commit ecd5d9f)
@sfc-gh-fpawlowski
[Async] Apply #2307 to async code
f18e51d
@sfc-gh-zyao @sfc-gh-fpawlowski
SNOW-2057867 refactor BindUploadAgent to make it work for Python spro…
da8553e 
…cs (#2303)

(cherry picked from commit 0d79989)
@sfc-gh-fpawlowski
[Async] Apply #2303 to async code
083d741 
# Conflicts:
#	src/snowflake/connector/aio/_direct_file_operation_utils.py
@sfc-gh-fpawlowski
Review fixes - made _ret iterable for aio
59f476d
@sfc-gh-fpawlowski
Review fixes - async mock method
847bb8c
@sfc-gh-pbulawa @sfc-gh-fpawlowski
SNOW-2111939: Bind cryptography to latest known working version (#2325)
f1b81f0 
(cherry picked from commit a6bd3cf)
@sfc-gh-zyao @sfc-gh-fpawlowski
SNOW-2057867 refactor and fixes to make pandas write work for Python … (
d0d8cf6 
#2304)

(cherry picked from commit fe9547b)
@sfc-gh-mkubik @sfc-gh-fpawlowski
Filter out Deprecation warnings from test_incalid_conection_parameter… (
c4d2876 
#2330)

(cherry picked from commit f1657d2)

# Conflicts:
#	test/integ/test_connection.py
@rcleveng @sfc-gh-mkubik @sfc-gh-fpawlowski
fix error message when SF_AUTH_SOCKET_ADDR was set (#2332)
be7faa4 
Co-authored-by: Mikołaj Kubik <mikolaj.kubik@snowflake.com>
(cherry picked from commit 07230cf)
@sfc-gh-fpawlowski
[Async] Apply #2332 to async code
76e7917
@sfc-gh-mkubik @sfc-gh-fpawlowski
SNOW-1947479 Add bulk_upload_chunks parameter to write_pandas (#2322)
fc2e513 
(cherry picked from commit c53aad7)
@sfc-gh-mmishchenko @sfc-gh-fpawlowski
SNOW-1762538 add detecting running inside a Jupyter notebook for coll…
0ace1a5 
…ecting usage stats (#2290)

(cherry picked from commit 54ad220)
@sfc-gh-fpawlowski
[Async] Apply #2290 to async code
92802e6
@sfc-gh-mmishchenko @sfc-gh-fpawlowski
NO-SNOW fix olddriver test by pinning version of pytest-cov (#2357)
b22fb9a 
(cherry picked from commit 56fffab)
@sfc-gh-pczajka @sfc-gh-fpawlowski
Fix #2290 async implementation
7dfde67
@sfc-gh-mmishchenko @sfc-gh-fpawlowski
NO-SNOW rename integration tests subfolders to avoid module import am…
13cef8d 
…biguity (#2364)

(cherry picked from commit 0641261)
@sfc-gh-fpawlowski
[Async] Apply #2364 to async code
9a9c61c
sfc-gh-fpawlowski and others added 18 commits November 27, 2025 08:43
@sfc-gh-fpawlowski
No snow proxy precence fix (#2615)
18acf7b 
(cherry picked from commit ec14974)

# Conflicts:
#	src/snowflake/connector/connection.py
@sfc-gh-fpawlowski
[async] Applied #2615 to async code
ab6b746
@sfc-gh-turbaszek @sfc-gh-fpawlowski
NO-SNOW: Send telemetry when closing connection (#2607)
2a5eaea 
(cherry picked from commit c14dff9)
@sfc-gh-fpawlowski
[async] Applied #2607 to async code
2417efb
@sfc-gh-turbaszek @sfc-gh-fpawlowski
SNOW-2194055: Separate server and redirect URIs in AuthHttpServer (#2609
7086967 
)

Co-authored-by: Nikita Peshkov <nikita.peshkov@teampicnic.com>

(cherry picked from commit 1a0290d)
@sfc-gh-fpawlowski
[async] Applied #2609 to async code
c82a269
@sfc-gh-fpawlowski
[async] Applied #2609 to async code
1f5282a
@sfc-gh-fpawlowski
SNOW-2462946: document credential caching (SSO, MFA) in the in-repo d…
e0702d0 
…ocs (#2626)

(cherry picked from commit 1a6c193)
@sfc-gh-fpawlowski
NO-SNOW: connect method API tests added (#2628)
0b43112 
(cherry picked from commit e54936b)
@sfc-gh-zyao @sfc-gh-turbaszek @sfc-gh-fpawlowski
SNOW-2483517 sproc continuous integration (#2629)
18ca45f 
Co-authored-by: Tomasz Urbaszek <tomasz.urbaszek@snowflake.com>
(cherry picked from commit a2162d8)
@sfc-gh-fpawlowski
[async] Applied #2628 to async code
ebd7c50
@sfc-gh-fpawlowski
[async] test fixes:
5140aed 
Fixed synch factory in async code of _wif.py
Fixed async make_session and added necessary host_port_pooling
@sfc-gh-fpawlowski
[async] fix windows jobs
934e184
@sfc-gh-zyao @sfc-gh-fpawlowski
SNOW-2483517 avoid test build recipe issue with recently available GC…
22658f9 
…C 15.x.x (#2644)

(cherry picked from commit 975d57c)
@sfc-gh-zyao @sfc-gh-fpawlowski
SNOW-2483517 Further improve test build recipe to isolate impact from…
f3f8b97 
… Jenkins env changes (#2646)

(cherry picked from commit 271e6ca)
@sfc-gh-fpawlowski
[async] fix timeout jobs
0b97e59
@sfc-gh-pczajka @sfc-gh-eworoshow
@sfc-gh-bli @sfc-gh-mkeller @sfc-gh-jszczerbinski @sfc-gh-jrose @sfc-gh-aling @sfc-gh-yixie @sfc-gh-dszmolka @Polandia94 @sfc-gh-rebeling @sfc-gh-mkubik @sfc-gh-fochnik @sfc-gh-pbulawa @sfc-gh-zli @sfc-gh-mmishchenko @sfc-gh-zyao @sfc-gh-tkissinger @sfc-gh-pmansour @sfc-gh-fpawlowski @sfc-gh-lspiegelberg @sfc-gh-akolodziejczyk @sfc-gh-mhofman @sfc-gh-pcyrek @sfc-gh-nkumar @rcleveng @sfc-gh-saroskar @sfc-gh-xizhao @sfc-gh-mborins @sfc-gh-skarve @sfc-gh-sshetkar @sfc-gh-yuwang @ozturkberkay @Argon- @khmyznikov @sfc-gh-hpathak @sfc-gh-gmerticariu @sfc-gh-eqin @sfc-gh-bchinn @jochenott @jochen-ott-by @sfc-gh-snoonan @sfc-gh-nkrishna @ericbuehl @mathiasflorin @sfc-gh-jkasten @sfc-gh-dheyman @HeroCC @sfc-gh-turbaszek @sfc-gh-jwilkowski @npeshkov
Merge main to aio (#2680)
1f29348 
Co-authored-by: Eric Woroshow <eric.woroshow@snowflake.com>
Co-authored-by: Bing Li <63471091+sfc-gh-bli@users.noreply.github.com>
Co-authored-by: Mark Keller <mark.keller@snowflake.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Jakub Szczerbiński <jakub.szczerbinski@snowflake.com>
Co-authored-by: Jamison <jamison.rose@snowflake.com>
Co-authored-by: Adam Ling <adam.ling@snowflake.com>
Co-authored-by: Yijun Xie <yijun.xie@snowflake.com>
Co-authored-by: David Szmolka <69192509+sfc-gh-dszmolka@users.noreply.github.com>
Co-authored-by: Pablo Nicolás Estevez <73709191+Polandia94@users.noreply.github.com>
Co-authored-by: Richard Ebeling <richard.ebeling@snowflake.com>
Co-authored-by: Mikołaj Kubik <mikolaj.kubik@snowflake.com>
Co-authored-by: Filip Ochnik <filip.ochnik@snowflake.com>
Co-authored-by: Piotr Bulawa <piotr.bulawa@snowflake.com>
Co-authored-by: Zihan Li <63482288+sfc-gh-zli@users.noreply.github.com>
Co-authored-by: Maxim Mishchenko <maxim.mishchenko@snowflake.com>
Co-authored-by: Zexin Yao <103003040+sfc-gh-zyao@users.noreply.github.com>
Co-authored-by: Thomas Kissinger <61704619+sfc-gh-tkissinger@users.noreply.github.com>
Co-authored-by: Peter Mansour <peter.mansour@snowflake.com>
Co-authored-by: Filip Pawłowski <filip.pawlowski@snowflake.com>
Co-authored-by: Leonhard Spiegelberg <leonhard.spiegelberg@snowflake.com>
Co-authored-by: Adam Kolodziejczyk <adam.kolodziejczyk@snowflake.com>
Co-authored-by: Michał Hofman <michal.hofman@snowflake.com>
Co-authored-by: Patryk Cyrek <patryk.cyrek@snowflake.com>
Co-authored-by: Naresh Kumar <113932371+sfc-gh-nkumar@users.noreply.github.com>
Co-authored-by: Rob Clevenger <rcleveng@users.noreply.github.com>
Co-authored-by: Susheel Aroskar <susheel.aroskar@snowflake.com>
Co-authored-by: Xiaohu Zhao <xiaohu.zhao@snowflake.com>
Co-authored-by: Myles Borins <myles.borins@snowflake.com>
Co-authored-by: Sanchit Karve <sanchit.karve@snowflake.com>
Co-authored-by: Sid Shetkar <sid.shetkar@snowflake.com>
Co-authored-by: Yuyang Wang <yuyang.wang@snowflake.com>
Co-authored-by: Berkay Öztürk <info@berkayozturk.net>
Co-authored-by: Julian <Argon-@users.noreply.github.com>
Co-authored-by: Gleb Khmyznikov <gleb.khmyznikov@gmail.com>
Co-authored-by: Harsh Pathak <harsh.pathak@snowflake.com>
Co-authored-by: George Merticariu <103256710+sfc-gh-gmerticariu@users.noreply.github.com>
Co-authored-by: Edward Qin <edward.qin@snowflake.com>
Co-authored-by: Brandon Chinn <brandon.chinn@snowflake.com>
Co-authored-by: jochenott <jochenott@users.noreply.github.com>
Co-authored-by: Jochen Ott <Jochen.Ott@blueyonder.com>
Co-authored-by: Sean Noonan <sean.noonan@snowflake.com>
Co-authored-by: Naren Krishna <naren.krishna@snowflake.com>
Co-authored-by: Eric Buehl <715650+ericbuehl@users.noreply.github.com>
Co-authored-by: Mathias Florin <31008003+mathiasflorin@users.noreply.github.com>
Co-authored-by: James Kasten <james.kasten@snowflake.com>
Co-authored-by: Dawid Heyman <dawid.heyman@snowflake.com>
Co-authored-by: HeroCC <2124489+HeroCC@users.noreply.github.com>
Co-authored-by: Tomasz Urbaszek <tomasz.urbaszek@snowflake.com>
Co-authored-by: Jakub Wilkowski <jakub.wilkowski@snowflake.com>
Co-authored-by: Nikita Peshkov <nikita.peshkov@teampicnic.com>
@sfc-gh-pczajka
Merge branch 'main' into dev/aio-connector
3e2c405
@sfc-gh-pczajka sfc-gh-pczajka marked this pull request as ready for review November 28, 2025 13:48
@sfc-gh-pczajka sfc-gh-pczajka requested a review from a team as a code owner November 28, 2025 13:48
@sfc-gh-pczajka sfc-gh-pczajka added DO_NOT_PORT_CHANGES_TO_SP Add this label when changes in this PR do not need to be port to SP connector and removed DO NOT MERGE labels Nov 28, 2025
@sfc-gh-pczajka sfc-gh-pczajka changed the title [DO NOT MERGE] Add support for async io Add support for async io Nov 28, 2025
sfc-gh-turbaszek
sfc-gh-turbaszek commented Nov 28, 2025
View reviewed changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sfc-gh-pczajka sfc-gh-pczajka sfc-gh-pczajka approved these changes

Assignees

No one assigned

Labels

DO_NOT_PORT_CHANGES_TO_SP Add this label when changes in this PR do not need to be port to SP connector

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.